ACADEMIC STANDARDS FOR 
WRITING 


To What Degree Do Standards Signpost Evidence-Based 
Instructional Practices and Interventions? 


ABSTRACT 

Though writing plays an important role in academic, 
social, and economic success, typical writing instruc- 
tion generally does not reflect evidence-based prac- 
tices (EBPs). One potential reason for this is limited 
signposting of EBPs in standards. We analyzed the 
content of writing standards from a representative 
sample of states and the Common Core State Stan- 
dards (CCSS) for writing and language to determine 
to what degree EBPs were signposted, variability of 
this signposting, and the overlap of practices sign- 
posted in states’ standards and the CCSS. We found a 
few practices signposted fairly consistently (e.g., iso- 
lated components of writing process instruction) and 
others rarely so (e.g., use of text models), as well as 
great variability across standards, with some covering 
almost half of the EBPs and others far fewer. Only a 
few states’ writing standards overlapped considerably 
with the CCSS. We discuss the implications of these 
findings for teacher professional development and for 
evaluating standards. 


A cademic writing is an essential part of the K-12 experience, as students 
are expected to compose texts to demonstrate, support, and deepen their 
knowledge and understanding of themselves, their relationships, and their 
world (Bangert-Drowns, Hurley, & Wilkinson, 2004; Graham, 2006; Gra- 
ham & Perin, 2007). Additionally, writing appears to be crucial for students’ success 
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on high-stakes achievement tests that have become a linchpin in school reform ef- 
forts in the United States, which have been motivated in part by global competitive- 
ness (e.g., Jenkins, Johnson, & Hileman, 2004; Reeves, 2000). Likewise, there is a 
growing trend to use writing proficiency as one determiner of graduation eligibility 
and in making decisions regarding grade retention and promotion (Zabala, Minnici, 
McMurrer, & Briggs, 2008). 

In postsecondary education, universities use writing to evaluate applicants’ qual- 
ifications, and proficient writing is expected for completion of a college degree (Na- 
tional Commission on Writing for America’s Families, Schools, and Colleges [NC- 
WAFSC], 2003, 2004, 2005). Once students leave an educational setting, writing 
serves as a gateway for employment and promotion (NCWAFSC, 2004). It is logical 
to conclude that, as the United States further transitions to an economy based in 
large part on information, technology, and services, the demands for proficient writ- 
ing in the workplace will continue to escalate (Bazerman, 2006). Of course, writing 
also serves many purposes in today’s civic life. In a nationally representative sample 
of 700 adolescents, 85% reported using some form of electronic personal communi- 
cation (e.g., text messages, social network posts, blogs, e-mails) for daily social in- 
teraction, self-exploration and expression, and reflection on current events (NC- 
WAFSC, 2008). Writing also may reduce psychological and physical distress and, 
consequently, health-care utilization (Harris, 2006). Together, these facts make the 
case for the central role of writing in society. 

Despite its importance for success as a lifelong learner and productive citizen, a 
large segment of the population struggles with writing: nearly three-quarters of the 
nation’s children and youth are not able to produce texts that are judged to fully meet 
grade-level expectations (National Center for Education Statistics, 2or2; Persky, 
Daane, & Jin, 2003; Salahu-Din, Persky, & Miller, 2008). Likewise, nearly a third of 
high school graduates are not ready for college-level composition courses (ACT, 
2007) and three-quarters of college faculty and employers rate their students’ and 
employees’ writing, respectively, as only fair or poor (NCWAFSC, 2004). One po- 
tential reason why so many individuals struggle with writing is the infrequent de- 
ployment of evidence-based instructional practices and interventions (EBPs) in 
many classrooms (e.g.. Burns & Ysseldyke, 2009). EBPs are a prima facie mechanism 
for boosting student achievement because they include methods, programs, or pro- 
cedures that integrate the best available research evidence with practice-based pro- 
fessional expertise in the context of student and family characteristics, values, and 
preferences (see American Psychological Association, 2005; Sackett, Straus, Richard- 
son, Rosenberg, & Haynes, 2000). Of course, the research evidence supporting par- 
ticular EBPs is often incomplete in terms of representing the full complement of 
student population characteristics and the entire range of educational contexts in 
which EBPs might be deployed. The use of an EBP derived from research, conse- 
quently, may not be the only or even the best course of action for every student in 
every circumstance (this is why research evidence must be integrated with other 
knowledge to teach effectively). Nevertheless, professional practice is founded upon 
a shared body of specialized knowledge, and EBPs comprise one sector of that body 
of knowledge in education. 

With respect to the implementation of EBPs for writing, self-report data from 
a national sample of elementary teachers show that instruction in planning, 
revising, and editing strategies for composing texts occurs less than 10 minutes a 


ACADEMIC STANDARDS FOR WRITING ’ 293 


day (Cutler & Graham, 2008). In secondary classrooms (see Applebee & hanger, 
2006, 2011; Kiuhara, Graham, & Hawken, 2009), teachers report frequently giv- 
ingwriting assignments that require little analysis, interpretation, or actual com- 
posing (i.e., abbreviated responses, worksheets) and devoting less than 3 hours 
per marking period to instruction related to writing strategies (and even less time 
to other aspects of instruction). A large percentage of primary grade teachers 
report making few or no adaptations for struggling writers (Graham, Harris, 
Fink-Ghorzempa, & MacArthur, 2003), and high school teachers report infre- 
quently adapting their teaching for lower-performing writers (Kiuhara et ah, 
2009). Data from classroom observation studies are generally discouraging as 
well, though there are certainly some excellent writing teachers who adopt many 
EBPs (e.g., Foorman & Schatschneider, 2003; Moats, Foorman, & Taylor, 2006; 
Rowan, Gamburn, & Gorrenti, 2004). 

Why are EBPs for writing not more widespread in U.S. classrooms? The lack of 
clear, coherent, and consistent research-based standards to help guide teachers’ in- 
structional efforts may be a culprit (Duke, 2001; Dutro & Valencia, 2004; Spillane, 
1998; Troia & Maddox, 2004). Academic standards are designed, ideally, to inform 
curriculum development, guide instruction and assessment, provide clear goals for 
student achievement, and raise performance expectations (e.g., Stecher, Hamilton, 
& Gonzalez, 2003). There is a limited body of scholarship that indicates improve- 
ments to states’ writing standards (and assessments) can positively influence class- 
room instruction. For example, in response to changes in their state’s writing stan- 
dards and high-stakes tests, teachers reportedly increased instructional emphasis on 
writing for specific audiences and purposes, at least those valued by the state’s tests 
(Hillocks, 2002; Stecher, Barron, Ghun, & Ross, 2000), writing across the curriculum 
(Applebee & Langer, 2011; Taylor, Shepard, Kinner, & Rosenthal, 2002) and the time 
allocated to daily writing (Stecher et ak, 2000). Nevertheless, the impact of these 
instructional changes on actual student writing performance was found to be negli- 
gible (Stecher et al., 2000). 

There are several plausible reasons why improvements to learning standards in 
the domain of writing would not translate to better student writing outcomes. First, 
assessments used for educational accountability, or high-stakes tests, which by their 
very nature only sample a portion of learning standards (often those that are readily 
measurable), may counteract benehts of enhanced standards by narrowing the writ- 
ing curriculum (e.g., Applebee & Langer, 2011; Hamilton, 2004; Hillocks, 2002; 
Stecher, 2002). Second, research (e.g., Kurz, EUiott, Wehby, & Smithson, 2010) sug- 
gests that the intended curriculum (prescribed by standards) often does not corre- 
spond to the enacted curriculum (i.e., what is actually taught, when it is taught, and 
how) or the learned curriculum (i.e., what knowledge, skills, abilities, and disposi- 
tions students attain). Moreover, many argue that standards should not prescribe 
but simply guide teaching anyway (Myers, 1994) . Third and key to the study reported 
here, standards may or may not emphasize particular instructional practices that 
positively impact student writing. Learning standards that do not support and shape 
the deployment of EBPs in classrooms may hinder the goal of raising student 
achievement because teachers are not directed toward these practices through the 
standards, and thus must rely on external sources for pedagogical knowledge. Un- 
fortunately, no studies have evaluated this aspect of standards. 
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In this study, we describe the degree to which writing standards, including the 
newly adopted Common Core State Standards for writing and language (CCSS- 
WL), “signpost” EBPs for writing. We use the term signpost to reflect the inter- 
connectedness of the language used in standards and the definitions of specific 
instructional practices that presumably could be employed to help students at- 
tain standards. As such, signposting implies a bidirectional relationship between 
the content of standards and classroom instructional practices — standards help 
shape classroom instruction and specific instructional practices can help stu- 
dents meet the standards. We also examine the extent to which EBPs signposted 
in state standards overlap with the practices signposted in the CCSS-WL, as 
substantial mismatch implies that Common Core adopting states will have much 
work ahead to develop teachers’ capacity to enact different practices not sign- 
posted in previous standards. Of course, academic standards are designed to 
explicate the “what” of instruction, not necessarily the “how.” Nevertheless, 
standards can and often do signpost for educators particular ways in which the 
standards can be attained via instructional practices. For example, a focus on 
writing process in a set of standards implies that educators must have students 
engage in the processes of planning, drafting, revising, editing, and publishing 
texts and consequently use a process-based approach to teach writing in at least 
some circumstances. Likewise, a call to provide guidance, support, and feedback 
in early elementary standards but not standards for later grades does, in fact, 
specify instructional action — in this case scaffolding. 

We had three research questions in this descriptive study: (i) What EBPs are 
signposted most and least in a purposive sample of standards, including the CCSS- 
WL? (2) What variability exists in EBP signposting across sets of standards and across 
grades? (3) To what degree do EBPs signposted in states’ standards align with those 
signposted in the CCSS-WL? These are salient research questions if we assume stan- 
dards affect classroom instructional practices and that certain practices are more 
likely to help students attain specific standards. If specific EBPs are signposted more 
often than others, this may communicate to teachers that greater value is accorded 
these practices and encourage them to use them and not others. Differences in EBP 
signposting across sets of standards and even across grades might be linked to vari- 
ations in instructional quality and coherence and student achievement. Obviously 
we are making an assumption that may or may not be valid: EBPs that are signposted 
are more likely to be enacted. This assumption requires empirical exploration and is 
not the goal of this study. Because we examined the presence/absence of EBPs in 
standards, the first step in our research (described below) was to identify what writ- 
ing instructional practices are, in fact, evidence based. We relied on published meta- 
analyses of writing instruction to accomplish this goal because meta- analysis affords 
the most reliable mechanism for identifying the efficacy and/or effectiveness of a 
particular practice. 


Method 

Evidence-Based Practices 

We conducted a thorough review of the PsychINFO and ERIC databases for quan- 
titative meta-analyses of studies examining writing instruction and assessment using 
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the title search terms writing, written, text, composition, composing, spelling, handwrit- 
ing, effect, synthesis, and meta-analysis. We also contacted the author most frequently 
associated with such meta-analyses, Dr. Steve Graham, to identify any in-press or 
other published meta- analyses. The search yielded 21 relevant citations from journal 
articles, book chapters, and dissertations (noted with an asterisk in the references). 
Of these, five reports of meta-analysis were excluded because they did not examine 
the impact of writing instruction or assessment practices on writing outcomes (Fri- 
sina, Borod, & Lepore, 2004; Graham & Hebert, 2011; Harris, 2006; Hebert, Simpson, 
& Graham, 2013; Smyth, 1998). Thus, we examined 16 meta-analyses to extract a list 
of EBPs for writing. 

Prior to extracting EBPs from the meta-analyses, each report of meta-analysis was 
evaluated for methodological rigor using an adapted version (available from the first 
author) of the Meta- Analysis Reporting Standards (MARS) of the American Psycho- 
logical Association (2008). We adapted the MARS in two ways; (1) desirable but 
nonessential standards for discerning methodological rigor were eliminated (e.g., 
title and abstract features), and (2) a three-point rating scale (o = absent, 1 = par- 
tially present, 2 = fully present) was added to permit determination of the degree to 
which each reporting standard was met. The scale included 38 standards, yielding a 
total score between 0 and 76 for each meta-analysis. The standards evaluated the 
following key features of meta-analyses: (a) empirical and theoretical grounding and 
analytic rationale, {h) primary study inclusion and exclusion criteria, (c) moderator 
and mediator analyses, (d) search strategies, (e) primary study coding procedures, if) 
data reduction and statistical modeling, (g) results reporting, and (h) discussion of 
generalizability, implications, and limitations. Three trained graduate student raters 
(the third, fourth, and fifth authors) independently scored each of the meta-analyses 
for methodological rigor. The two-way mixed-effects intraclass correlation for mean 
ratings was .98. Given the high degree of scoring interrater reliability (IRR), we used 
the mean score assigned for methodological rigor. We established a threshold score 
of 38 to consider a meta-analysis minimally suitable for our purposes; consequently, 
two meta-analyses (Graham & Harris, 2003; Schramm, 1991) were considered no 
further due to low scores. 

Provided in Table 1 are (a) the citation for each meta-analysis from which we 
extracted EBPs, (b) the mean score for methodological rigor using the adapted 
MARS, (c) the mean effect sizes for writing-related outcomes associated with distinct 
practices reported in each meta-analysis, (d) the grades in which the primary re- 
search associated with each practice was conducted, and (e) the definition we ad- 
opted for each practice. Definitions were based on those provided in the meta- 
analyses and in source studies, though we did reclassify or combine some practices 
for the sake of parsimony (e.g., peer vs. adult feedback in Graham, McKeown, Kiu- 
hara, & Harris, 2012; prewriting activities vs. planning and drafting instruction in 
Rogers & Graham, 2008). Additionally, some practices reported in the meta-analyses 
were not included in Table 1 because they did not relate to standards in any obvious 
way (e.g., free writing and individualized tutorials/programmed materials in Hill- 
ocks, 1984; teacher reinforcement in Rogers & Graham, 2008) or demonstrated neg- 
ative or negligible effects on writing outcomes (e.g., traditional grammar instruction 
in Graham & Perin, 2007 and Hillocks, 1984). 

Following review of these meta- analyses, we developed a list of EBPs to search for 
within standards based on a content coding framework we previously developed for 


Atkinson (1993) 3, 12, college Process writing Instruction that involves (1) writing for real/authentic/ Writing quality .52 63.00 
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6, 8, 11 -college Transcription skills Focus on process of teaching students spelling, Writing quality 

instruction handwriting, and keyboarding (typing) skills to improve 

quality of writing. 
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Setting Product Goals Teachers or students set observable, specific, and Writing quality 

individual goals for what students are to accomplish in 
their writing such as how much students should write; 
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Graham & Sandmel 1-12 Process writing Writing quality .34 71-67 

(2011) instruction 

Hillocks (1984) Elementary-college Process writing Writing quality .18 56.67 
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examining writing standards and assessments (Troia et al., submitted), which is 
described below. Thus, the EBPs were linked with existing standard content codes 
and, in some cases, additional key words or phrases required by the definition of the 
EBP. For instance, the EBP of teacher and/or peer feedback was signposted by a 
content code of feedback (verbal or written information in response to an author’s 
work at any point in the writing process received from peers and/or adults). In 
contrast, the EBP of sentence-combining instruction was signposted by the presence 
of the content codes associated with sentence fluency (the variety, appropriateness, 
and use of sentences in the text), general grammar (a general reference to the struc- 
ture of language at the word and/or sentence levels), or specific grammar (the inten- 
tional application of specific morphosyntactic elements or rules, such as specific 
parts of speech, sentence fragments and run-ons, verb tense and agreement, phrase 
and clause structures, etc.) plus the keyword “combining” or “rearrange.” The list of 
28 EBPs and the required content codes and keywords/phrases are provided in Table 
2 (note that two EBP categories were further subdivided based on our content coding 
and thus there are 28 EBPs in total — teaching prewriting/planning/drafting was di- 
vided into three subcategories and transcription skills instruction also was divided 
into three subcategories). Because EBPs are directly linked to the linguistic content of 
standards (which was organized by categorical strands), they are listed in the order 
they correspond to the content coding strands (e.g., the first five EBPs listed relate to 
the standards coding content strand of writing processes). 

As can be seen in this table, many of the coding decisions for EBPs were based on 
what might be best described as logic statements. For instance, the definition for 
process writing instruction includes three essential features: (1) writing for real/au- 
thentic/multiple purposes and audiences; (2) engaging in cycles of planning, trans- 
lating, and reviewing; and (3) personal responsibility and ownership of writing proj- 
ects. Thus, this EBP could be signposted only if the content of a set of standards for 
a grade or grade band referred to (a) the general writing process or a combination of 
planning, drafting, and revising and (b) both purpose and audience and (c) self- 
regulation or the keyword “choice.” Omission of even one of these parts of the logic 
statement would exclude the assignment of the EBP of process writing instruction to 
the set of standards. 

Sample of Writing Standards 

We selected a purposive sample of states to examine the degree to which their K- 8 
writing standards signposted EBPs and to compare them with the CCSS-WL. We 
selected 14 states (see Table 3) that represented diversity with respect to geographical 
location, population density, and performance of eighth graders on the 2007 Na- 
tional Assessment of Educational Progress (NAEP) state-level writing assessment. In 
addition, we limited our sample to states that had not altered their writing standards 
within the 4 school years prior to the 2007 NAEP administration (assuming that 
stability in standards would create less “noise” in the relationship between a state’s 
standards and its associated performance on the NAEP). NAEP performance data 
and population data for each of these states, as well as length of time their standards 
were in force prior to the fall 2007 administration of the NAEP writing test, are 
presented in Table 3. 
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Standards Content Coding 

As is typical for standards content analysis research (e.g., Porter, 2002), our coding 
taxonomy employs broad content strands (i.e., categories) to designate major in- 
structional and developmental foci in writing as well as specific content indicators 
within each strand that provide categorical elaboration. The taxonomy was derived 
from several theoretical frameworks to assure a broad representation of current 
thinking about writing development, instruction, and assessment. Specifically, we 
drew upon Hayes’s cognitive model of writing (Flower & Hayes, 1981; Hayes, 1996) to 
develop two content strands — (1) writing processes and (2) metacognition and 
knowledge; sociocultural theory (Prior, 2006) — to form the (3) context strand; genre 
theory (Dean, 2008) to inform two strands — (4) purposes and (5) components; and 
linguistic models of writing (Faigley & Witte, 1981) — to create the (6) conventions 
strand. The last strand — (7) motivation — was inspired by both cognitive and moti- 
vation theories of writing (Troia, Shankland, & Wolbers, 2012). We included an 
eighth strand to capture information within standards that was not clearly related to 
writing but appeared to be potentially relevant to writing and connected to one of the 
other strands. We also included a single code for information within standards that 
appeared to reflect important information but that did not fit any of the coding 
categories. This coding taxonomy, containing 112 separate content indicators across 
the strands, allowed us to differentiate sets of standards in terms of their content 
using linguistically based micro-analysis. While other writing theories could have 
been applied, they made no additional contributions for the purpose of coding con- 
tent in the standards. Content coding employed the content indicators for a fine- 
grained analysis rather than the strand categories, which would have yielded insuf- 
ficient detail. 

An individual content indicator code was applied within a unit of content analysis 
(i.e., the lowest consistent level of content information in a set of standards) only 
once to avoid duplication, but multiple different codes could be assigned to any given 
unit. To accommodate the potential for additional information presented at higher 
levels of organization for a set of standards, unique codes were assigned at these 
superordinate levels, but duplication of codes from the lower levels was not allowed. 
Thus, a state that only used two levels of organization for its standards could be 
compared to another state that used four levels without bias being introduced by the 
specific organizational pattern chosen by a state. Additional information regarding 
our content coding taxonomy and procedures can be found in Troia et al. (submit- 
ted). 


Coding Reliability 

Content coding for the CCSS-WL and the sampled states’ writing standards was 
completed by two trained graduate student raters (the third, fourth, or fifth authors). 
The standards for grades 1 and 6 were coded and resolved first before coding and 
resolving differences in the remainder of the K-8 standards for each set to ensure 
adequate reliability. This process was necessary due to the number of judgments 
needed to accurately interpret the content of each state’s writing standards. All dis- 
crepancies were resolved through discussion. The content coding IRR ranges across 
grades and medians (prior to reconciliation) were as follows; CCSS-WL range .76 to 
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20004 Peer collaboration Students cooperatively work with their peers to plan, draft, revise, and/or edit their [Collaboration] 

compositions. 

20005 Providing extra time for Duration and/or frequency of sustained student writing are increased (e.g., write [Disciplinary context OR writing in/out of school OR 
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Table 3. State Standards Sampled, Demographics, and 2007 NAEP Performance 


State 

Region 

2010 Population 
(Millions) 

2007 NAEP Writing 
Scale Score ^ 

Years Standards in 
Force Prior to 
Fall 2007 

AL 

Southeast 

4.8 

148 j, 

8 

AR 

Southeast 

2.9 

151 i 

4 

CA 

West 

37-3 

148 i 

10 

CO 

West 

5.0 

161 1 

7 

FL 

Southeast 

18.8 

158 T 

11 

IN 

Midwest 

6.5 

155 = 

7 

MA 

Northeast 

6.5 

167 T 

6 

MN 

Midwest 

5-3 

156 = 

4 

MS 

Southeast 

3-0 

142 J, 

5 

NV 

West 

2.7 

142 J, 

6 

OH 

Midwest 

11.5 

156= 

5 

TN 

Southeast 

6-3 

156 = 

6 

TX 

Southwest 

25.1 

151 i 

9 

WI 

Midwest 

5-7 

158 T 

9 


Symbols indicate state scale score is significantly above ( f ), significantly below ( f ), or statistically equivalent (=) to national 
sample scale score average of 154. 


.95 (median of .89); AL range .81 to .97 (median of .88); AR range .71 to .94 (median 
of .80); CA range .77 to .97 (median of .88); CO range .85 to .97 (median of .96); FL 
range .70 to .88 (median of .83); IN range .74 to .95 (median of .88); MA range .75 to 
.87 (median of .79); MN range .86 to .95 (median of .89); MS range .88 to .95 (median 
of .91); NV range .76 to .99 (median of .93); OH range .80 to .95 (median of .93); TN 
range .72 to .91 (median of .84); TX range .84 to .99 (median of .96); and WI range .97 
to .99 (median of .98) . The reliability for assignment of EBPs across grades K-8 was 
evaluated for six of the states sampled (AL, AR, CA, MN, TX, WI) and the CCSS-WL; 
the mean exact agreement between two trained graduate student raters (the third and 
fourth authors) ranged from 96% to 100%. 


Results 

Reported in Table 4 are the evidence-based writing instruction practices signposted 
by standards from the sample of 14 states and the CCSS-WL for K-2, 3-5, and 6-8 
grade bands. The pluses indicate the practice was signposted in the standards at least 
once within the grade band. Because in many cases a practice was, in fact, signposted 
only in one or perhaps two grades in the band, the range of EBPs signposted in each 
grade band also is reported. For instance, AL has 14 total practices signposted at least 
once in grades K-2, but the range of practices signposted is 4 (in kindergarten) to 14 
(in second grade). Consequently, the range of EBPs signposted is more representa- 
tive of the consistency with which they are signposted. Florida and Wisconsin do not 
exhibit a range because these states had standards for each grade band rather than 
unique standards for each grade; in other cases where there is no range, it is because 
signposting of EBPs was consistent across the three grades. As a point of reference, 
the greatest range occurs for TX in grades K-2 (8-19 EBPs signposted). The percent- 
age of EBPs signposted in each grand band (averaged across the three grades) is 
reported as a summary statistic. We also calculated for each grade band for each state 
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Handwriting instruction 
Keyboarding instruction 
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EBP Signposted across K-2 Standards EBP Signposted across 3-5 Standards EBP Signposted across 6-8 Standards 
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a CCSS-WL alignment index — the proportion of EBPs in the CCSS-WL also sign- 
posted by the state’s standards. Finally, the last portion of Table 4 includes the per- 
centage of grade cells (out of a total of 41 grade cells for the 15 sets of standards; recall 
that FL and WI collapse standards across grades) in which each FBP was signposted 
across all standards we evaluated for each grade band; this provides a summary 
statistic representing the degree to which a particular practice was signposted by 
grade band. 

There are several patterns evident when examining Table 4. First, the EBPs sign- 
posted by most states’ standards and the CCSS-WL at most grades include (a) exe- 
cuting the various aspects of the writing process — teaching prewriting (mean of 
81%), planning (mean of 88%), drafting (mean of 74%), and editing (mean of 
89%) — though not in the context of integrated process-based writing instruction, as 
demonstrated by no signposting for process writing instruction; (b) spelling instruc- 
tion (mean of 94%); (c) using a word processor to compose (mean of 77%); and (d) 
inquiry-based writing instruction (mean of 77%). Conversely, the EBPs signposted 
least in the group of standards we studied across grades include (a) integrated 
process-based writing instruction (mean of 0%; note that process writing instruction 
is not coded when combined with other types of writing instruction to form a com- 
prehensive writing instruction program, so the percentages should be interpreted 
accordingly — see Table 2); (h) using writing as a means to learn content area infor- 
mation (mean of 2%); (c) setting written product goals (mean of 4%); (d) strategy 
instruction (mean of 12%); (e) using assistive technology (mean of 13%); if) key- 
boarding instruction (mean of 13%); ig) sentence-combining instruction (mean of 
15%); (h) using text models (mean of 21%); and (i) self-regulation of writing and 
writing metacognition (mean of 27%). 

Second, there is a general trend of greater signposting of EBPs in third through 
eighth grade compared with kindergarten through second grade (ranging from an 
increase of 5% for setting product goals to an increase of 47% for paragraph structure 
instruction), most likely reflecting increased writing content coverage in the stan- 
dards for older students across states and the CCSS-WL. This general trend is re- 
versed for handwriting, spelling, and strategy instruction. In 10 cases the percentage 
of EBPs signposted in middle school grades 6-8 is lower than in upper elementary 
grades 3-5; for four EBPs this drop is substantial (handwriting instruction — 37%, 
summarization instruction — 15%, paragraph structure instruction — 15%, decreas- 
ing spelling errors through editing — 15%). More typical is a higher percentage of 
EBPs signposted in middle school grades than in the upper elementary grades. For 
three EBPs this increase is substantial (text structure instruction — 14%, teaching 
prewriting — 15%, inquiry-based instruction — 25%). 

Third, the variability with which EBPs are signposted tends to be substantially 
greater in grades K-2 (mean range difference of 4.7) than grades 3-5 (mean range 
difference of 2.3) or grades 6-8 (mean range difference of 0.9). Thus, consistency of 
EBP signposting within grade bands increases across grade bands. Fourth, based on 
the percentages of EBPs signposted across grades for the sets of standards we evalu- 
ated, CA, CO, MS, and WI have the least signposting of EBPs in their standards, with 
means of 31%, 26%, 28%, and 32%, respectively. Conversely, AR, TN, and TX have 
the greatest signposting, with means of 63%, 61%, and 67%, respectively. 

Finally, with respect to alignment with the CCSS-WL, AR (mean alignment index 
of .93) and TX (mean alignment index of .88) exhibit the strongest alignment overall. 
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while CO (mean alignment index of .56), MS (mean alignment index of .52), and WI 
(mean alignment index of .51) exhibit the weakest. In most cases, alignment with the 
CCSS-WL increases across grade bands. Alignment and the percentage of EBPs sign- 
posted are strongly related: r = .75, p < .01 for grades K-2, r = .83, p < .01 for grades 
3-5, and r = .69, p < .01 for grades 6-8. 


Discussion 

Standards-based education reform is predicated on the assumption that rigorous 
academic standards (and the high-stakes assessments aligned with them) exert a 
powerful positive influence on classroom instruction. Although academic standards 
are intended primarily to guide instruction via specification of what major compe- 
tencies students should demonstrate at a given grade in a given domain of learning, 
standards also might guide instruction via signposting specific practices (e.g., Cohen, 
1996). To our knowledge, no other study has attempted to identify the degree to 
which standards, for writing or any other domain of learning, signpost EBPs. It is 
clear from our findings that standards do indeed signpost specific writing instruction 
practices and that there is variability in the degree to which different standards do 
this. 

Specifically, we found three states (AR, EL, TN) in our sample that consistently 
signposted in each grade K-8 at least 12 of the 28 EBPs for which we coded; another 
two states (OH, TX) did so at most of those grades. Only TN consistently signposted 
more than half of the EBPs for which we coded. Conversely, in each grade we con- 
sidered, CO, MS, and WI consistently signposted less than 12 of the practices. The 
CCSS-WL signposted between 9 and 14 EBPs in a given grade. EBP signposting might 
prove to be a valuable new measure for evaluating the rigor of academic standards; 
however, as we noted earlier, EBP signposting may not necessarily lead to effective 
instructional practices and associated increases in student achievement (see Beck, 
2007, about the potential disconnect between the imprecise nature of standards and 
the classroom). This remains an open research question and ultimately will deter- 
mine the value of measuring EBP signposting in standards. A more plausible impli- 
cation of our findings is that in cases where standards don’t signpost many practices, 
there is probably a greater burden on professional development to introduce such 
practices to teachers — the practices themselves cannot be inferred or extracted from 
the standards and thus educators must rely on external sources (e.g., professional 
development) for their content and pedagogical knowledge. 

As for alignment with the CCSS-WL, AR and TX displayed the highest degree of 
overlap in EBPs signposted — teachers in AR, a CCSS-adopting state, may require less 
effort to transition to the new standards than say teachers in MS or WI, where the 
degree of overlap is only around 50%. In these adopting states with limited overlap, 
teachers may not see the relevance of particular instructional practices they may have 
previously used for implementing the new CCSS-WL (e.g., summarization instruc- 
tion) or they may have to learn about new practices that are signposted in the 
CCSS-WL but absent from their state’s prior writing standards (e.g., peer collabora- 
tion). Professional development efforts will need to focus on transitioning from one 
set of standards to the other by identifying shared content and signposted EBPs as 
well as discrepancies along these dimensions, providing implementation guidelines 
for specific practices, and identifying relevant curriculum materials that are aligned 
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with the content of the CCSS-WL and that reflect EBPs signposted in the core stan- 
dards. Of course, professional development also should make salient those EBPs that 
are not signposted in the CCSS-WL. 

We also found in the group of standards we evaluated that most EBPs are sign- 
posted with greater frequency in later grades, save for basic transcription skills in- 
struction (spelling and handwriting) and strategy instruction, which were signposted 
more often in grades K-2. It is evident that states’ standards relay a message to 
teachers that transcription skills should be mastered early, which reflects the impor- 
tance of automaticity in basic transcription for freeing cognitive resources for more 
complex composing operations (e.g., Berninger & Amtmann, 2003). Moreover, 
standards point to a greater number of discrete EBPs over time (and do so more 
consistently), probably because they add greater content breadth in each grade, con- 
tent that is generally maintained once introduced. In the domain of writing, this may 
be advantageous because it mimics the transition from novice to competent (and 
perhaps expert) writer, in which topic and domain knowledge and skills are acquired 
over time and these funds of knowledge and skill become more automated, inte- 
grated, and flexible to permit the writer to attack novel writing tasks with more 
complex and deeper strategic approaches, though less complex and superficial ap- 
proaches can be applied when they suffice (e.g., Alexander, 2003; Bereiter & Scarda- 
malia, 1993)- 

Particular practices, such as teaching elements of the writing process, spelling 
instruction, and using word processing software, were signposted more often than 
others, such as an integrated process approach to teaching writing, using assistive 
technology to support the writing process, and keyboarding instruction. These hnd- 
ings suggest some tension in what is communicated through standards. For instance, 
although using the computer to write is valued in many sets of standards, keyboard- 
ing instruction, without which many students would struggle to word process their 
papers (e.g., Graham et al., 2012), is not. Likewise, although spelling instruction 
appears to be valued in many standards, instruction in the use of assistive technology 
(e.g., spell check, speech recognition) that might help students bypass spelling diffi- 
culties or aid in better spelling (e.g., MacArthur, 2006) does not appear to be valued. 
It is important that those groups charged with developing standards recognize the 
importance of consistency and coherence in the content of standards and the values 
expressed therein, as well as the impact of the messages standards convey to educa- 
tors, students, and families. 


Study Limitations 

This study, like any other, has limitations that should temper any conclusions 
drawn from the information presented. First and foremost, one cannot presume that 
signposting of EBPs or other instructional practices in standards leads to corre- 
sponding classroom teaching actions. There are far too many other intervening vari- 
ables, such as teachers’ experiences, values, beliefs, and attitudes regarding writing, 
writing instruction, and standards (e.g., Graham, Harris, Fink, & MacArthur, 2001; 
Graham, Harris, MacArthur, & Fink, 2002; Lipson, Mosenthal, Daniels, & 
Woodside-Jiron, 2000), the quality and quantity of professional development and 
knowledge in this domain (e.g.. Cutler & Graham, 2008; Gilbert & Graham, 2010; 
Pritchard & Honeycutt, 2006; Troia & Maddox, 2004), and the curriculum materials 
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used to enact the standards that exert an influence on classroom writing instruction. 
But, as we have argued above, the degree of signposting of EBPs in standards does 
have implications for how schools address these intervening variables. 

Second, we based our analysis on EBPs derived from high-quality meta- 
analyses, considered the “gold standard” for determining the efficacy (and po- 
tentially the effectiveness) of discrete interventions and instructional routines. 
Other instructional practices that improve writing performance have empirical 
support (e.g., vocabulary instruction, conferencing, the use of authentic and 
relevant writing tasks), but were not included in the meta-analyses we examined 
because too few individual quantitative studies examined these practices to be 
synthesized through meta-analytic techniques to derive reliable effects estimates. 
Thus, our list of EBPs, while clearly based on the best available research evidence, 
is not exhaustive. Standards may very well have signposted other valuable writing 
instruction practices for which we did not code. Conversely, it is important to 
remain cognizant of the limits of EBPs for prescribing writing instruction for all 
students — the research undergirding EBPs does not include every different kind 
of learner (e.g., students with disabilities and/or gifts and talents, English lan- 
guage learners) in every potential educational context (e.g., inner city school, 
small-group remedial sessions, English immersion classes) in every grade for 
every kind of writing (e.g., poetry, informative papers, procedures). Simply put, 
an EBP designation does not guarantee a practice will be successful with every 
learner in every situation, so flexibility with integrating professional expertise, 
research evidence, and student and family preferences and needs to help each 
student attain grade-level writing content standards remains paramount. 

Third, we analyzed standards from a relatively small, though representative 
sample of states. We were limited to (a) states from which we could collect 
standards documents and (b) states that had not subjected their standards to 
major revision several years prior to the 2007 NAEP writing assessment. State 
performance on the 2007 NAEP versus the 2011 NAEP writing test was used as a 
benchmark for selecting standards to include in our evaluation because state- 
level writing performance data are only available for the 2007 assessment. Our 
data only reflect EBP signposting for the state standards we sampled and cannot 
be generalized to the entire nation. Moreover, six of the states (AL, AR, EL, IN, 
MS, NV) revised their standards after 2006 but prior to CCSS adoption; we did 
not evaluate these revised standards, which may have aligned more or less with 
the core standards than their previous ones. 


Conclusion 

Many educators prefer to rely on the advice of successful teachers of writing (e.g., 
Atwell, 1998) or professional authors (e.g.. King, 2000) to guide their own writing 
instruction because they mistrust educational research, which is the foundation of 
EBPs (e.g., Boardman, Arguelles, Vaughn, Hughes, & Klingner, 2005; Jones, 2009). 
They may also rely more often on professional advice because there is not a compre- 
hensive writing instruction research base. Researchers can help inform teachers and 
other practitioners about the benefits of using rigorous replicated research to select 
teaching practices that will help students attain academic standards (and the plausi- 
ble constraints of the research for employing a practice with a particular student) and 
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the limitations of practices derived from isolated case studies and professional wis- 
dom, particularly those practice recommendations that fall outside the list of EBPs 
for a domain or that counter prevailing EBPs. However, a potentially more expedi- 
tious and potent strategy for enhancing the credibility of EBPs in the classroom may 
be to point educators to these practices through education policy and accountability 
mechanisms (e.g., academic standards, teacher credentialing standards, high-stakes 
assessments) with well-aligned professional development efforts. The effects of pro- 
fessional development might be greatly enhanced if standards included companion 
materials that link the standards with EBPs and provide concrete implementation 
guidelines. 

Research is needed to determine the validity of the degree of EBP signposting for 
evaluating standards by way of determining whether variability in this measure is 
reliably associated with teachers’ application of EBPs in their classroom instruction. 
Such research will have to accommodate the complex interplay of the characteristics 
of standards with other education production functions such as professional devel- 
opment, high-stakes assessments, available classroom resources, and teacher quality. 
Additionally, the meditational effects of teachers’ knowledge, beliefs, and values on 
instruction as they enact the intended curriculum should be studied. Likewise, 
teacher interpretation of standards, which is not well understood, is probably an- 
other important mediator to examine. Of course, efforts to examine the impact of 
standards on classroom instruction should look toward the salutary effects on stu- 
dent achievement. 
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